Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new splitting command #116

Merged
merged 23 commits into from
Jul 12, 2024
Merged

add new splitting command #116

merged 23 commits into from
Jul 12, 2024

Conversation

anjor
Copy link
Contributor

@anjor anjor commented Jul 1, 2024

No description provided.

@anjor anjor marked this pull request as draft July 1, 2024 12:19
@anjor anjor force-pushed the anjor/car-subset-split branch from 14ec563 to 78afc84 Compare July 3, 2024 11:51
@anjor anjor marked this pull request as ready for review July 5, 2024 12:51
@anjor anjor requested a review from gagliardetto July 5, 2024 12:51
@anjor anjor mentioned this pull request Jul 5, 2024
2 tasks
@gagliardetto
Copy link
Collaborator

it would be useful to add some more context to the errors using fmt.Errorf("error while doing this thing for this other thing: %w", err)

@anjor
Copy link
Contributor Author

anjor commented Jul 8, 2024

@gagliardetto added. Any other feedback?

cmd-car-split.go Outdated Show resolved Hide resolved
cmd-car-split.go Outdated Show resolved Hide resolved
@gagliardetto
Copy link
Collaborator

After the changes, please try out the new split command on a car file (early epochs should be good size).

After that, use faithful-cli dump-car /path/to/split-123.car on each split piece. It should print everything without errors.

@anjor
Copy link
Contributor Author

anjor commented Jul 8, 2024

$ yellowstone-faithful split-car -e 0 -s 1000000000 ./epoch-0.car

split it into the following 5 files:

$ ls -l *car
-rw-r--r--  1 anjor  staff  1004138599 Jul  8 11:16 epoch-0-1.car
-rw-r--r--  1 anjor  staff  1004146286 Jul  8 11:17 epoch-0-2.car
-rw-r--r--  1 anjor  staff  1004146195 Jul  8 11:17 epoch-0-3.car
-rw-r--r--  1 anjor  staff  1004145672 Jul  8 11:18 epoch-0-4.car
-rw-r--r--  1 anjor  staff   269251520 Jul  8 11:18 epoch-0-5.car
-rw-r--r--@ 1 anjor  staff  4286945461 Jul  8 11:09 epoch-0.car

dump-car didn't throw any errors. Just printed out a bunch of stuff.

cmd-car-split.go Outdated Show resolved Hide resolved
cmd-car-split.go Outdated Show resolved Hide resolved
cmd-car-split.go Outdated Show resolved Hide resolved
cmd-car-split.go Outdated Show resolved Hide resolved
cmd-car-split.go Show resolved Hide resolved
cmd-car-split.go Outdated Show resolved Hide resolved
cmd-car-split.go Outdated Show resolved Hide resolved
anjor and others added 2 commits July 11, 2024 16:58
@gagliardetto
Copy link
Collaborator

LGTM

@gagliardetto gagliardetto merged commit 874f38e into main Jul 12, 2024
3 checks passed
@anjor anjor deleted the anjor/car-subset-split branch October 29, 2024 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants